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Claim Rejections - 35 USC§112 
The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

Claims 2-10, 12-13, 15, 23, 25-26, 29-31, 33-34, 37-39, and 45-47 rejected under 35 
U.S.C. 1 12, first paragraph, as failing to comply with the enablement requirement. The claim(s) 
contains subject matter, which was not described in the specification in such a way as to enable 
one skilled in the art to which it pertains, or with which it is most nearly connected, to make 
and/or use the invention. 

Regarding the independent claims 12, 17, 22, 29, and 37: 

Re. claim 12 at third line, the terms "a height field" is not defined in the specification to 
represent how does the height of the surface compute at each point? 

Re. claim 17, the language of "a viewpoint separate from viewpoints associated with the 
multi image sensors" is not clear how the viewpoint is separated from viewpoints associated with 
the multi image sensors in the specification. 

Re. claim 22 at line five similar arguments as argued in claim 12. 

Re. claim 29. at lines 30-31 uses "... a temporal pixel average of five recent image 
frames. . that is not defined in the specification to clear the five points as a Stencil or something 
else. 

Re. claim 37 at lines 32-33 similar arguments as argued in claim 29. 
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Claim Objections 

The numbering of claims is not in accordance with 37 CFR 1.126 which requires the 
original numbering of the claims to be preserved throughout the prosecution. When claims are 
canceled, the remaining claims must not be renumbered. When new claims are presented, they 
must be numbered consecutively beginning with the number next following the highest 
numbered claims previously presented (whether entered or not). 

Claim 16 is depending to itself; it should be dependent to claim 15. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 2-10, 12-13, 15, 23, 25-26, 29-31, 33-34, 37-39, and 45-47 rejected under 35 

U.S.C. 103(a) as being unpatentable over Frederick Weinhaus and Venkat Devarajan with title of 

"Texture Mapping 3D Models of Real- World Scenes" ACM Vol 29, No.4 1997 pp.325-363, 

(hereinafter refers as Frederick), and further in view of Moura et al. US 6,760,488B1, 

(hereinafter refers as Moura). 

Claim 12. 

Moura in fig. 4 steps 32, 38 and 42 illustrates the following claim languages: A method 
comprising: generating a three dimensional model of an environment from range sensor 
information representing a height field for the environment; Moura at col. 2 lines 37-38 the 
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system includes: an image sensor for capturing a sequence of two-dimensional images of a 
scene. 

Moura at col. 3, lines 4-6 teaches the invention uses a parametric description of the shape 
and the induced optical flow parameterization which has advantages over the prior arts tracking 
feature points that may be unreliable when processing noisy video sequences. As a result, to 
alleviate this situation in the prior art, it is known to assume a very short interval between frames 
for easy tracking. See following part of the claim: tracking orientation information of at least one 
image sensor in the environment with respect to the three-dimensional model in real-time; Moura 
at col. 22 lines 2-8 teaches that the 3D models obtained from the video data according to the 
present invention may be used to build a synthetic image sequence. This synthesis is achieved 
by specifying the sequence of viewing positions along time. The user specifies the viewing 
positions, either in an interactive way or from an automatic procedure. 

The next step of the claim i.e. projecting real-time video imagery information from the at 
least one image sensor onto the three-dimensional model based on the tracked orientation 
information; Moura at cols. 1-2, lines 66-67 and 1-11, respectively, teaches according to one 
known technique which does not require computing of an estimate of the absolute depth as an 
intermediate step, the 3D positions of the feature points are expressed in terms of Cartesian 
coordinates in a world-centered coordinate system, and the images are modeled as orthographic 
projections . The 2D projection of each feature point is tracked along the image sequence. The 
3D shape and motion are then estimated by factorizing a measurement matrix whose entries are 
the set of trajectories of the feature point projections. The factorization of the measurement 
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matrix, which is rank 3 in a noiseless situation, is computed by using a Singular Value 
Decomposition (SVD) expansion technique. 

Moura in figs. 12a and 13a illustrates visualizing the three-dimensional model with the 
projected real-time video imagery. 

Moura for the following section of the claim "wherein projecting the real-time video 
imagery information comprises generating a depth map image from a video sensor viewpoint, 
and projective texture mapping the real-time video imagery information onto the three 
dimensional model conditioned upon visibility as determined from the generated depth map 
image" clearly in fig. 4 and at col. 26 lines 5-22 teaches the feature points were tracked by 
matching the intensity pattern of each feature along the sequence. Using the rank 1 weighted 
factorization described hereinbefore, the 3D motion and the relative depth of the feature points 
was recovered from the set of feature trajectories. Fig. 15 show two perspective views of the 
reconstructed 3D shape with the texture mapped on it. The angle between the walls is clearly 
seen and the round edge is also reconstructed. 

Moura is silenced about the limitations in next part of the claim, however, Frederick at 
second column second paragraph on page 349 teaches the claim limitations a An SGI computer 
was used to perform the rendering by simply interpolating the color values across the triangles 
and by using a depth-buffer to remove hidden pixels. See part of the claim's limitations: wherein 
generating the depth map image and projective texture mapping the real-time video imagery 
information are performed using a one-pass approach on graphics hardware that supports SGI 
OpenGL extensions. 

Frederick is silenced to have one image sensory comprises multi image sensors. 
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Thus, it would have been obvious to one ordinary skill in the art at the time the invention 
was made to combine Moura's fig. 17 step 24 into Frederick's texture mapping for an efficient 
and less computationally-intensive technique to recover 3D structure from a 2D image sequence. 
Claim 2. 

Frederick on page 355 at first col. teaches that in this system, 3D models composed of 
parametric primitives such as blocks, pyramids, and the like are manually composited to 
approximate the geometry of the architectural structure using the imagery as a visual guide. 
Claim 3. 

Frederick in fig. 3c illustrates the claim limitations. 
Claim 4. 

Frederick on page 356 at second col. teaches that retrieving the most appropriate oblique 
perspective image stored in the video disk bank and then warp it according to Equation (1) to 
simulate the pilot's or sensor's view of the gaming area at that instance in time. Since these 
operations were accomplished in real-time, it would give the illusion of smooth interactive 
motion through the gaming area. 
Claim 5. 

Frederick on page 342 at first col. teaches the height-field approach also has been applied 
to the urban setting where the elevations of the terrain posts are raised locally to account for the 
heights of buildings. Frederick on page 344 at the bottom of first col. teaches under section 3.2.2 
Point Projection Approaches for Height Fields. 
Claim 6. 
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Frederick on page 349 at second col. teaches in Hughes' approach, only those terrain 
elevation tiles that were found to lie at least partially within the output image's ground 
"footprint" (projection onto the X, Y ground plane) were rendered. The terrain elevation model 
and imagery resolutions were selected as a function of distance between the terrain elevation tile 
and the eye point so that coarse resolution could be used for more distant regions. Terrain 
elevation tiles were then tessellated into triangular meshes at the specified resolution. 
Claim 7. 

Frederick on page 355 at first col. teaches it may even be moved along the ray from the 
eye point to the object. Shadow mensuration is an option that can be used to help in 3D model 
construction. Also, shadows can be rendered into a graphic overlay on the transformed view. 
Semi-automated 2D feature extraction tools, such as road following and the detection of 
boundaries of buildings whereby the user starts the extraction process with a manual cue, also are 
included. SRI also has experimented with automatic extraction of 3D models of buildings and 
used their system to portray the results. 
Claims 8-9. 

Frederick on page 327 at first col. under section (4) teaches the claim limitation. 
Frederick on page 326 at second col. teaches Volotta interactive video has put together a system, 
called the Mars Navigator. 
Claim 10. 

Frederick on page 340 at the bottom of first col. teaches multiple views. 
Claim 13. 
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Frederick on page 352 at second col. teaches a combination of automatic stereo 
compilation followed by manual editing can be used to generate the terrain elevation model. 
Claim 22. 

See rejection of claim 12 that applies to the rejection for claim 22. 
Claim 15. 

Frederick on page 340 at the bottom of first col. teaches multiple views. 
Claim 16. 

Frederick on page 360 at first col. teaches the claim limitation. 
Claim 17. 

The claim limitation is obvious because when generating a 3D image, the viewpoint of 
the video imagery projection should be separate from viewpoints from multiple image sensors. 
Claim 18. 

Frederick in fig. 8 illustrates the claim limitation. 
Claim 19. 

Frederick on page 342 at first col. teaches the height-field approach also has been applied 
to the urban setting where the elevations of the terrain posts are raised locally to account for the 
heights of buildings. Frederick on page 344 at the bottom of first col. teaches under section 3.2.2 
Point Projection Approaches for Height Fields. Frederick on page 349 at second col. teaches in 
Hughes' approach, only those terrain elevation tiles that were found to lie at least partially within 
the output image's ground "footprint" (projection onto theX, Aground plane) were rendered. 
The terrain elevation model and imagery resolutions were selected as a function of distance 
between the terrain elevation tile and the eye point so that coarse resolution could be used for 
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more distant regions. Terrain elevation tiles were then tessellated into triangular meshes at the 
specified resolution. 
Claim 20. 

Frederick in fig. 8 illustrates the claim limitation. 
Claim 21. 

Frederick on page 355 at bottom of first and top of the second columns teaches a novel 
view-dependent texture-mapping technique is used to render the architectural model. Multiple 
photographs are projected onto the model in order to texture its surface completely. (An image- 
space shadow-mapping algorithm based on a z-buffer is used to keep track of obscurations.) 
However, since the photographs overlap, the rendering algorithm must decide which photograph 
or photographs to use at each output pixel. 
Claim 23. 

Frederick on page 339 at first col. teaches in addition to showing transformed images in 
perspective, panoramic, and orthographic formats, they also presented a pair of transformed 
images that could be viewed stereoscopically in 3D. Frederick on page 355 at bottom of first and 
top of the second columns teaches a novel view-dependent texture-mapping technique is used to 
render the architectural model. Multiple photographs are projected onto the model in order to 
texture its surface completely. (An image-space shadow-mapping algorithm based on a z-buffer 
is used to keep track of obscurations.) However, since the photographs overlap, the rendering 
algorithm must decide which photograph or photographs to use at each output pixel. 
Claim 29. 

See rejection of claim 12 that applies to the rejection for claim 29. 
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Claim 25. 

Frederick in fig. 8 illustrates the claim limitation. 
Claim 26. 

Frederick in fig. 4 illustrates the claim limitations. 
Claim 30. 

Frederick on page 355 at second col. teaches a weighted average of the textures from the 
overlapping images is used, where the weights are the angular deviations of the viewing vectors 
of each source image from that of the output view. Moreover, to avoid visible "seams," the 
weights are ramped near the boundaries of source photographs. Optionally, a model-based stereo 
correlation algorithm may be used to refine the 3D model to include finer detail such as recessed 
windows, and the like, and to increase the fidelity of the texture-mapped rendering. 
Claim 31. 

Frederick in fig. 8 illustrates the claim limitation. 
Claim 37. 

See rejection of claim 12 that applies to the rejection for claim 37. 
Claim 33. 

Frederick in fig. 8 illustrates the claim limitation. 

Claim 34. 

Frederick in fig. 4 illustrates the claim limitations. 
Claim 38. 
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Frederick on page 327 at first col. under section (4) teaches the claim limitation. 
Frederick on page 326 at second col. teaches Volotta interactive video has put together a system, 
called the Mars Navigator. 
Claim 39. 

Frederick on page 342 at first col. teaches the height-field approach also has been 
applied to the urban setting where the elevations of the terrain posts are raised locally to account 
for the heights of buildings. Frederick on page 344 at the bottom of first col. teaches under 
section 3.2.2 Point Projection Approaches for Height Fields. Frederick on page 349 at second 
col. teaches in Hughes' approach, only those terrain elevation tiles that were found to lie at least 
partially within the output image's ground "footprint" (projection onto the X, Aground plane) 
were rendered. The terrain elevation model and imagery resolutions were selected as a function 
of distance between the terrain elevation tile and the eye point so that coarse resolution could be 
used for more distant regions. Terrain elevation tiles were then tessellated into triangular meshes 
at the specified resolution. 
Claim 45. 

Frederick in table 1 teaches the claim limitation. 
Claim 46. 

Frederick on page 338 teaches the selected portions of some of these frames 
corresponding to face-on views of building facades were mapped onto the faces of 3D models of 
the buildings. Distant mountains were also textured to add a sense of realism. Computer 
synthesized images simulating driving down the streets as well as low altitude views were then 
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generated in segments and stored on video disk for later playback under joystick control. During 
playback, turns could only be initiated at selected locations such as street intersections. 
Claim 47. 

Frederick in table 1 teaches the claim limitation. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A. Amini whose telephone number is 571-272-7654. The 
examiner can normally be reached on 8-4pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on 571-272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Javid A Amini 
Examiner 
Art Unit 2628 

Javid Amini 




